Video transmission method, video processing device, and video generating system for virtual reality

ABSTRACT

The disclosure provides a video transmission method for virtual reality. The method includes reorganizing a first video and a second video obtained to generate a third video suitable for transmission through a physical wire, hence avoiding distortion caused by compression of a high-definition video. The disclosure further includes a video processing device and a video generating system employing the video transmission method.

FIELD OF THE INVENTION

Embodiments of the disclosure relate to a video transmission method, avideo processing device and a video generating system for virtualreality, and particularly, to a method, device and system for videoprocessing in response to limitations of a transmission format beforethe use of a physical wire for transmitting a high-definition video,hence preventing a high-definition video needed for virtual reality frombeing limited by the transmission format of the physical wire and henceavoiding distortion inevitably caused by compression.

BACKGROUND

The development of wireless mobile network technology has provided moreopportunities for the use of greater transmission bandwidths and morediverse application services. Virtual reality (VR) is a result of thedevelopment combining techniques including computer graphics, computersimulation, artificial intelligence, sensing, display and networkparallel processing, and uses a three-dimensional virtual world toprovide users a comprehensive and perceptible artificial environmentincluding auditory and haptic perception, in a way that an immersivevisual perception in such environment is produced to allow directobservation, manipulation and touching of the ambient environment andinner changes of things and hence interaction, giving users the sense ofreality. By combining video streaming developed under real-timehigh-speed wireless mobile networking and the VR technology, athree-dimensional virtual world is capable of presenting a real scene inreal time, offering users enjoyment of realistic immersive experiencewithout being at the scene.

SUMMARY OF THE INVENTION

A video transmission method for virtual reality is provided according toan embodiment of the disclosure. The video transmission method includessteps of: obtaining a first video and a second video, both the firstvideo and the second video having a 2:1 aspect ratio; segmenting thesecond video from a short side of the second video to generate two firstsub videos having a same aspect ratio; segmenting the first sub videosfrom a long side of one of the first sub videos to generate three secondsub videos having a same aspect ratio; generating a third video, whereinthe third video includes the first video, the non-segmented first subvideo and the three second sub videos, and a short side of the thirdvideo is formed by a short side of the first video and a short side ofthe non-segmented first sub video, and the other short side of the thirdvideo is formed by three short sides of the three second sub videos,such that the third video has a 16:9 aspect ratio; and transmitting thethird video through a physical wire.

A video processing device for virtual reality is provided according toanother embodiment of the disclosure. The video processing deviceincludes a video input mechanism, a video processing mechanism and avideo output mechanism. The video input mechanism is configured toreceive a first original video and a second original video obtained froma first camera device. The video processing mechanism is in transmissionconnection to the video input mechanism, and includes a video processingunit configured to adjust the first original video and the secondoriginal video to a first video and a second video, and the videoprocessing unit is configured to further combine the first video and thesecond video into a third video having a 16:9 aspect ratio. Dimensionsof the third video are a total of those of the first video and thesecond video. The video output mechanism is in transmission connectionto the video processing mechanism, and is for outputting the third videothrough a physical wire.

A video generating device for virtual reality is further providedaccording to yet another embodiment of the disclosure. The videogenerating device includes a first camera device, a video processingdevice and a streaming encoder. The video processing device is intransmission connection to the first camera device, and is configured toadjust a plurality of original videos obtained by the first cameradevice to a plurality of virtual reality videos, wherein the virtualreality videos are combined to generate a transmission video having a16:9 aspect ratio. The streaming encoder is in transmission connectionto the video processing device through a physical wire, and isconfigured to convert the transmission video into a streaming signal.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the disclosure are described in detail with theaccompanying drawings below for better understanding. It should be notedthat, the structures are not drawn to scale according to a standardpractice in the industry. In fact, to provide the clarity of thedisclosure, sizes of the structures may be increased or decreased asdesired.

FIG. 1 is a structural schematic diagram of a video generating systemaccording to some embodiments of the disclosure.

FIG. 2 is a structural schematic diagram of a video processing deviceaccording to some embodiments of the disclosure.

FIG. 3 is a schematic diagram of a combination of a first video and asecond video according to some embodiments of the disclosure.

FIG. 4A is a schematic diagram of resolutions of a first video and asecond video according to some embodiments of the disclosure.

FIG. 4B is a schematic diagram of resolutions of a segmented secondvideo according to some embodiments of the disclosure.

FIG. 4C is a schematic diagram of a resolution of a third videoaccording to some embodiments of the disclosure.

DETAILED DESCRIPTION

The disclosure below provides various different embodiments or examplesof different components used to implement the subject matter of thedisclosure. Specific examples of components and configurations are givenin the description below to simplify the disclosure. It should be notedthat these components and configurations are exemplary and are notintended to be restrictive. For example, in the description below, afirst component formed on or above a second component may also includean embodiment in which the first component and the second component areformed in a direct contact manner, and may include an embodiment inwhich an additional component is formed between the first component andthe second component in a way that the first component and the secondcomponent may not be in direct contact. Moreover, numerals and/oralphabetical symbols may be repeated in the various embodiments of thedisclosure. Such repetition is intended for simplicity and clarity anddoes not represent relations between the embodiments and/orconfigurations.

Further, for better description, relative spatial terms such as “below”,“under”, “underneath”, “above”, “on”, and “over” and the like may beused to describe the relation between one element or component and otherelement(s) or component(s) as given in the drawings. In addition to theorientation depicted in the drawings, the relative spatial terms arealso intended to cover different orientations of a device in use or inoperation. An apparatus may be orientated by other means (rotated by 90degrees or having another orientation), and descriptive relative spatialterms used herein may also be similarly interpreted.

As used herein, terms such as “first”, “second” and “third” are used todescribe various elements, components, regions, layers and/or sections,and these elements, components, regions, layers and/or sections are notto be restricted by these terms. These terms are used to differentiateone element, component, region, layer or section from another element,component, region, layer or section. Unless otherwise specificallyindicated in the context, the terms such as “first”, “second” and“third” do not imply a sequence or order when used herein.

In some embodiments of the disclosure, virtual reality, and moreparticularly, virtual reality combining the real-time streamingtechnique, is established, to ensure that a virtual reality videoexperienced by a user has high definition (for example, meeting thedefinition standard that is at least 6K). Accordingly, a videoprocessing method for current hardware environments is provided toensure that a real-time video thus obtained is, without compromising anyvideo quality, transmitted by combing different means including aphysical wire and network streaming and then accurately presented at auser end, offering the user with quality feelings of presence.

As shown in the structural diagram of a video generating system in FIG.1 , in some embodiments, at least one first camera device 101 may bearranged to obtain a dynamic real-time video serving as at least a partof specific video contents restored in a virtual space created at a userend by the virtual reality technology. In some embodiments, the firstcamera device 101 is a camera having at least two lenses. For example,the first camera device 101 may include horizontally arranged dualfisheye lens to simulate the left and right eyes of a human and thedistance therebetween, so as to obtain videos with respect to differentfields of view of the simulated left and right eyes of the human. Thevideos are combined and processed by image processing techniques togenerate a three-dimensional video having a sense of depth.

In some embodiments, the first camera device 101 has a function ofcapturing a 180-degree video, and is capable of generating a 180-degreethree-dimensional video using visible fields of view of the simulatedleft and right eyes of the human in combination with the horizontallyarranged dual fisheye lens above, so as to establish a virtualenvironment in a three-dimensional video.

In some embodiments, the video generating system may further include atleast one second camera device 102. The second camera device 102 hashardware specifications identical to those of the first camera device101, and so the second camera device 102 is also capable of capturing a180-degree video. Further, in some embodiments, the first camera device101 and the second camera device 102 may provide different scenarioangles. For example, in live broadcast of an art and culturalperformance activity or a sport competition event, the first cameradevice 101 and the second camera device 102 are arranged at differentpositions of a venue of the activity or the competition, so as toprovide different scenario angles for observation and experience of auser. In another embodiment, a combination of the first camera device101 and the second camera device 102 may provide a 360-degree video;that is, the 180-degree videos respectively obtained by the first cameradevice 101 and the second camera device 102 are combined into a360-degree video. When more second camera devices 102 are provided, morecombinations of two of the second camera devices 102 may also be used toobtain a 360-degree video captured at different capturing positions.

To enhance the sense of reality of virtual reality, resolutions oforiginal videos generated by the first camera device 101 and the secondcamera device 102 need to be in certain levels. In some embodiments, thefirst camera device 101 and the second camera device 102 need to be atleast capable of outputting 4K resolution or higher. In someembodiments, high-resolution videos provided by the first camera device101 and the second camera device 102 have a 50 fps frame rate or more.In some embodiments, the first camera device 101 and the second cameradevice 102 have at least a chroma sampling specification of 4:2:2. Insome embodiments, the first camera device 101 and the second cameradevice 102 can support a color depth specification up to 10-bits. Insome embodiments, in order to match with limitations of softwareoperation performance or specifications of other hardware systems, thefirst camera device 101 and the second camera device 102 may provideother color depth specifications that can be supported during the actualoperation, for example, providing videos having a 8-bit color depth.

In order to achieve the object of processing a high-resolution videointo a video complying with a virtual reality standard (or referred toas a VR video), as shown in FIG. 1 , in some embodiments, the firstcamera device 101 and the second camera device 102 are in transmissionconnection to a video processing device 200 to perform virtual realityprocessing on original videos obtained by the camera device(s). In someembodiments, the video processing device 200 is deployed on a real siteof live broadcast of an art and cultural performance activity or sportcompetition event. In other words, for a virtual reality user (orreferred to as a spectator), the first camera device 101, the secondcamera device 102 and the video processing device 200 are all“remote-end” apparatuses, and the virtual reality user can acquire animmersive experience using a “near-end” virtual reality device throughhigh-speed, high-bandwidth and low-delay communication techniques, henceachieving a live broadcast mode of virtual reality. For a serviceprovider of virtual reality live broadcast, processes includingacquisition of high-resolution videos, virtual reality processing of thehigh-resolution videos and rendering customized special effects to thevideos are completed by the “near-end” first camera device 101, secondcamera device 102 and video processing device 200 deployed on a realsite. Then, the videos of the real site are provided for real-timeexperience to the virtual reality user at a “remote end” of the serviceprovider by the real-time streaming communication technique with no needto be present in the real site, further at the same time allowing thevirtual reality user to enjoy added functions such as special effects,freely switching of fields of view as desired, and viewing ofcompetition data and results.

Since the first camera device 101, the second camera device 102 and thevideo processing device 200 are all located at a near end of the serviceprovider of the virtual reality live broadcast, in some embodiments, thecamera devices including the first camera device 101 and the secondcamera device 102 perform data transmission with the video processingdevice 200 through physical wires. In some embodiments, the cameradevices and the video processing device 200 are connected by a SerialDigital Interface (SDI) 150 in between. In some embodiments, the SDI 150may be in the form of a board and has four input ports, and may thus beconnected to one first camera device 101 and one second camera device102, that is, left-eye and right-eye field-of-view videos (101L and101R) of the first camera device 101 as well as left-eye and right-eyefield-of-view videos (102L and 102R) of the second camera device 102 areinputted to the video processing device 200 through different inputports of one SDI 150. In some other embodiments, assuming that eightcamera devices (one first camera device 101 and seven second cameradevices 102) are used for example, original videos obtained by thecamera devices are inputted to the video processing device 200 usingfour SDIs 150. In some embodiments, videos captured by camera devicessuch as the first camera device 101 and the second camera device 102have 4K definition standard, and are outputted in the form of SDIsignals having a 3840*2160 resolution and a 16:9 aspect ratio to thevideo processing device 200. However, the disclosure does not in factlimit the specific specification of the SDI 150, and appropriatehardware apparatuses may be used according to the development ofcommunication transmission interfaces.

In some embodiments, the first camera device 101, the second cameradevice 102 and the video processing device 200 include transmission suchas SDI signals using coaxial cables in between. In some embodiments, thecamera devices and the video processing device 200 include optical fibercables and/or SDI-optical converters in between; that is, the cameradevices and the video processing device 200 may also performtransmission through optical fiber cables or a combination of opticalfiber cables and coaxial cables. In some other embodiments, videotransmission may also be performed using a more economicalHigh-Definition Multimedia Interface (HDMI) or DisplayPort (DP) andcorresponding HDMI wires and DP wires.

In some embodiments, the video processing device 200 has a function ofprocessing an original video in real time into a virtual reality video.The first camera device 101 and the second camera device 102 do not needto be virtual reality cameras (that is, the first camera device 101 andthe second camera device 102 do not belong to virtual reality cameradevices), and so this means that the camera devices do not need to havea function of directly outputting virtual reality videos. Therefore, inthe present disclosure, first of all, the virtual reality processing isperformed by the video processing device 200, and thus a high costcaused by a large number of virtual reality camera devices deployed on areal site can be avoided. Secondly, a virtual reality camera device isusually capable of achieving virtual reality videos having coarsequality as being limited by hardware specifications thereof, which doesnot meet the requirement for the level of definition. Moreover, if avirtual reality camera device outputting a virtual reality video isemployed in the system, it means that it may be difficult to performcustomized post-processing and editing of the virtual reality video.Therefore, the first camera device 101 and the second camera device 102employed in the present disclosure are not virtual reality cameradevices, instead, they are used to obtain original videos, and thenpost-processing and virtual reality processing of all videos arecompleted by using the video processing device 200.

In some embodiments, the video processing device 200 is a switcherhaving a function of performing virtual reality processing on a video.In some embodiments, as shown in FIG. 2 , the video processing device200 for virtual reality of the disclosure includes a video inputmechanism 201, a video processing mechanism 202 and a video outputmechanism 203. In some embodiments, the video input mechanism 201 is forreceiving a first original video and a second original video (forexample, the left-eye field-of-view video 101L and the right-eyefield-of-view video 101R that have not yet undergone virtual realityprocessing) obtained by the first camera device 101. If the videoprocessing device 200 is in transmission connection to at least onesecond camera device 102, the video processing device 200 may alsoreceive a third original video and a fourth original video (for example,the left-eye field-of-view video 102L and the right-eye field-of-viewvideo 102R that have not yet undergone virtual reality processing)obtained by the second camera device 102. Similar to the first originalvideo and the second original video which originate from differentfields of view of the simulated left and right eyes, the third originalvideo and the fourth original video also correspond to different fieldsof view based on the same principle.

Due to the possibility of multiple camera devices used, in someembodiments of the present disclosure, the video input mechanism 201includes an input management unit (not shown). Thus, in addition to thevideo from the first camera device 101, switching between differentvideo sources may be performed using the input management unit, forexample, switching to receive the third original video and the fourthoriginal video obtained from the second camera device 102.

In some embodiments, by a first video processing unit 204 of the videoprocessing mechanism 202, the first original video and the secondoriginal video can be adjusted to a first video 301 and a second video302 (to be described with reference to FIG. 3 ) that meet virtualreality requirements. In some embodiments, the first video processingunit 204 is a graphics processing unit (GPU). For example, adjustmentperformed by the first video processing unit 204 of the video processingmechanism 202 on the original videos may include video stitching. Asdescribed above, the first video device 101 may capture a 180-degreevideo, and so in some embodiments, the left-eye field-of-view video 101Land the right-eye field-of-view video 101R captured by the first cameradevice 101 are 180-degree videos from different fields of view, and theleft-eye field-of-view video 102L and the right-eye field-of-view video102R captured by the second camera device 102 are 180-degree videos fromdifferent fields of view. In some embodiments, the left-eyefield-of-view videos 101L and 102L in a 180-degree video specificationare stitched into the first video 301 having a 360-degree videospecification, and the right-eye field-of-view videos 101R and 102R in a180-degree video specification are stitched into the second video 302having a 360-degree video specification. The first video 301 and thesecond video 302 are 360-degree videos which include two 180-degreevideos (not a continuous 360-degree panoramic video but a 360-degreevideo having contents of two 180-degree videos). After being transmittedto a virtual reality device, the first video 301 and the second video302 are respectively projected onto left-eye and right-eye displaymechanism of the virtual reality device, so as to offer a user with asense of three-dimensional virtual reality video. It should be notedthat, the first video and the second video of the disclosure are notlimited to being corresponding to the left-eye field of view and theright-eye field of view, the description above merely provides examples,and the correspondence between the two may be swapped.

In some embodiments, the video processing mechanism 202 may include andoperate a renderer 206, which renders a special effect to the firstvideo 301 and the second video 302. The special effect may includenumerous aspects. For example, in order to pass the sense of thrill andexcitement of a real site to a virtual reality user, the first video 301and the second video 302 may be rendered with a special effect for thepurpose of enhancing visual effects (VFX), such as providing differentand customized special effects with respect to art performances,concerts or sport competition events. Moreover, the special effectrendered may also include data analysis information provided by a thirdparty, such as contents including weather information, game scores ofsport competition events, competition analysis, player's data andinformation and sport scientific information which are rendered to thefirst video 301 and the second video 302, allowing a virtual realityuser to experience three-dimensional virtual reality videos and tobrowse images to obtain more information that help enhance the effectsof experience.

In some embodiments, the video processing mechanism 202 may include andoperate or access a virtual reality media database 208 so as to obtainvideo materials and to feed in videos including common on-site livebroadcast videos, advertisement videos, close-ups, slow motions andhighlight playbacks. In some embodiments, the virtual reality mediadatabase 208 may also provide customized styles such as differentsubtitles, flashcards or backgrounds as well as animation effects, so asto easily superimpose in combination with the function of the firstvideo processing unit 204 of the video processing mechanism 202 one ormore layers onto the first video 301 and the second video 302.

In some embodiments, the first video processing unit 204 of the videoprocessing mechanism 202 may perform video processing procedures such ascolor correction, brightness and contrast adjustment, dimming, positionadjustment, truncation and scaling on the first video 301, the secondvideo 302, special effects to be rendered or video contents added. Insome embodiments, a virtual reality scene may also be captured usingonly one camera device, and real-time background removal may beperformed using the first video processing unit 204 or further incombination with the function of the renderer 206, so as to achieve agreen screen video synthesis function in the virtual reality scene.

As described above, to enhance the sense of reality of virtual reality,the resolutions of the original videos generated by the first cameradevice 101 and the second camera device 102 need to be in certainlevels, for example, having at least 4K definition standard, and atleast a 3840*2160 resolution. The video that has been processed by thefirst video processing unit 204 of the video processing mechanism 202,as shown in FIG. 3 , for example, the first video 301 and the secondvideo 302 formed by stitching, may have at least a 5760*2880 resolution,and the first video 301 and the second video 302 may be combined into avirtual reality video 30 having a 1:1 aspect ratio and a resolutionformat that can be used by the virtual reality device (the first video301 and the second video 302 are synchronously displayed on left-eye andright-eye display mechanisms of the virtual reality device). Moreover,in some embodiments, the virtual reality video 30 has at least a5760*5760 resolution, that is, having at least 6K resolution. If theresolution of the virtual reality video 30 is inadequate, for example,the long sides of the first video 301 and the second video 302 includedare less than 5760 pixels and under 6K resolution accordingly,blurriness may be resulted in a visual environment of the virtualreality user to severely restrain the sense of experience of the virtualreality user. However, under current standard specifications, a physicalwire for outputting to a downstream streaming encoder does not supportthe resolution format having a 1:1 aspect ratio. Therefore, thedisclosure provides a video transmission method for virtual reality thatoutputs the virtual reality video 30 without affecting the videoquality.

Specifically, in some embodiments, the video processing device 200includes the video output mechanism 203, which is in transmissionconnection to the video processing mechanism 202 and is for outputtingthrough a physical wire a video that has been processed by the videoprocessing mechanism 202. In addition, before the video processingdevice 200 outputs the virtual reality video 30 that is then convertedto a streaming signal, transmission through a physical wire first needsto be performed. The physical wire may include the SDI 150 andcorresponding SDI cables that support up to 8K resolution.Alternatively, in other embodiments, the HDMI or DP and correspondingcables that support up to 8K resolution may be included as well. Takingthe SDI as a transmission interface commonly used for high-qualityvideos for example, although a physical wire including the SDI 150 andthe corresponding SDI cables can support 8K resolution and thus meetrequirements for transmitting the high-resolution virtual reality video30, such physical wire supports only a video format having a 16:9 aspectratio and is incompatible with the virtual reality video 30 having a 1:1aspect ratio. That is to say, in the practice of high-quality videotransmission, the resolution specification of a video may be restrainedby the standard supported by the transmission interface used. Thus, howto perform high-quality video transmission using a transmissioninterface such as the SDI restrained by a transmission specificationwhile meeting the requirements of high-quality video transmissionoutside existing transmission specifications is one task to be resolvedin the related technical field. In one comparative example, in view thatthe SDI 150 supports only a video format having a 16:9 aspect ratio, thevideo processing device adjusts by means of lossy compression thevirtual reality video 30 having a 1:1 aspect ratio to a resolutionformat having a 16:9 aspect ratio, for example, compressing the virtualreality video 30 from a 5760*5760 format to a 5760*3240 format, and thenthe 5760*3240 format of the compressed virtual reality video 30 isrestored by a virtual reality device at the user end to a 5760*5760format having a 1:1 aspect ratio. However, in this comparative example,a part of video information is lost during the process of compressingthe virtual reality video 30 from a 1:1 aspect ratio to a 16:9 aspectratio, and according to the Information Theory, the lost informationcannot be restored after the lossy compression. Thus, the resolution ofthe video restored by the virtual reality device at the user endinevitably is poorer than that of the virtual reality video 30 beforethe compression. The embodiment of the disclosure is capable of solvingthe problem above by first segmenting and reorganizing the virtualreality video 30 using the video processing device 200, while beingcompatible with existing physical wires supporting high-resolution videotransmission (e.g., including the SDI 150) standards. Associated detailsare given below.

Referring to FIG. 4A, the first video 301 and the second video 302generated by the first video processing unit 204 both have a 2:1 aspectratio, and would have been combined into the virtual reality video 30 ina resolution format having a 1:1 aspect ratio. However, in someembodiments of the disclosure, as shown in FIG. 4B, the second video 302is segmented from a short side of the second video 302 to generate twofirst sub videos 302 a having the same aspect ratio, and the first subvideos 302 a is segmented from a long side of one of the first subvideos 302 a to generate three second sub videos 302 b having the sameaspect ratio. Thus, as shown in FIG. 4C, a third video 303 can beformed. The third video 303 includes the first video 301, thenon-segmented first sub video 302 a, and three second sub videos 302 b.Moreover, a short side of the third video 303 is formed by a short sideof the first video 301 and a short side of the non-segmented first subvideo 302 a, and the other short side of the third video 303 is formedby three short sides of the three second sub videos 302 b, such that thethird video 303 has a 16:9 aspect ratio.

Taking a specific resolution for example, in the embodiment above, thefirst video 301 and the second videos 302 may have a 5760*2880resolution, and the two would have been combined into the virtualreality video 30 having a 5760*5760 resolution. However, as shown inFIG. 4C, after the segmenting and reorganizing of the embodiment above,the long side of the first sub video 302 a having a 5760*1440 resolutionis aligned with the long side of the first video 301 having a 5760*2880resolution, and the short sides of the three second sub videos 302 bhaving a 1920*1440 resolution are connected in common to the short sidesof the first video 301 and the first sub video 302 a. After thereorganizing, the third video 303 thus generated has a 7680*4320resolution, that is, a video having 8K resolution and a 16:9 aspectratio, and may be transmitted through a physical wire.

The third video 303 generated by the video processing device 200 of thedisclosure has an aspect ratio that is changed by means of segmentingand reorganizing, that is, only pixels of the video are rearranged andreorganized without performing any compression or restoration on thedefinition. Thus, the dimensions of the third video 303 transmitted by aphysical wire are a total of those of the first video 301 and the secondvideo 302.

In some embodiments, the step of segmenting the second video 302 so asto combine with the first video 301 into the third video 303 may becompleted by a shader 210 operated by the video processing mechanism202. The shader 210 may generate an instruction for the first videoprocessing unit 204 to process and modify the pixels of the first video301 and the second video 302. One benefit of using the shader 210 togenerate the third video 303 is that the first video processing unit 204is given an instruction in a low-level language featuringhigh-efficiency, thus preventing additionally using videopost-production software to perform video editing using a high-levellanguage and reducing operation performance depletion of the first videoprocessing unit 204.

In some embodiments, before the second video 302 is segmented, a specialeffect is rendered to the first video 301 and the second video 302 bythe renderer 206. In other words, the generating of the third video 303having a non-1:1 aspect ratio is a special process performed in responseto the specification limitations of a physical wire. Thus, before thethird video 303 is generated, virtual reality processing and adjustmentand addition of visual effects needed for the first video 301 and thesecond video 302 must have been completed.

In some embodiments, the segmented video may be the first video 301instead of the second video 302. In other words, given that either oneof the first video 301 and the second video 302 is segmented, thesegmented video may be reorganized with the other non-segmented video toform a video having a 16:9 aspect ratio and be transmitted through aphysical wire. Moreover, FIG. 4A to FIG. 4C of the disclosure showmerely examples of video segmenting and reorganizing forms, and do notimpose limitations on the sequence of steps in this disclosure regardingthe segmenting and reorganizing of the virtual reality video by theshader 210 in response to the limitations of the transmission format ofa physical wire.

Using the technical solution of segmenting and organizing a video asdescribed above, the disclosure skillfully arranges the virtual realityvideo 30 having a 1:1 aspect ratio into having a 16:9 aspect ratio. As aresult, this does not compromise the resolution of the virtual realityvideo 30, and at the same time fully utilizes the bandwidth of aphysical wire without producing any bandwidth waste in the transmissionprocess.

In some embodiments, the video processing mechanism 202 may include andoperate a user interface (UI). The UI is for an operating individual ofthe video processing device to select the source (for example, acombination of the first camera device and the second camera deviceabove, or a combination of two other second camera devices) of thevideo, so as to check the video and then confirm the visual effectrendered, the quality of the virtual reality processing, and operationssuch as selection and other related editing of videos.

As described above, the video output mechanism 203 is in transmissionconnection to the video processing mechanism 202, and outputs through aphysical wire the video completely processed by the video processingmechanism 202, wherein the video is the third video 303. In someembodiments, the video processing device 200 is a switcher having afunction of performing virtual reality processing on a video. In someembodiments, the video processing device 200 may be a machine groupconsisting of a plurality of computers connected to one another, whichperform different functions and process a video to be outputted.

As shown in FIG. 1 , in some embodiments, the video generating systemincludes a storage device 400. The storage device 400 is in transmissionconnection to the video processing device 200, and is for storing avideo (that is, the third video 303) outputted by the video processingdevice 200. The storage device 400 functions as a data backup, and insome other embodiments, the video generating system may exclude thestorage device 400.

In some embodiments, the video generating system includes a streamingencoder 500. The streaming encoder 500 is in transmission connection tothe video processing device 200 through a physical wire, and is forconverting the third video 303 outputted by the video processing device200 into a streaming signal. For example, the third video 303 having a16:9 aspect ratio may be encoded by the streaming encoder 500 into anHTTP live streaming (HLS) signal, which is then further transmitted to aremote end using an existing wired or wireless network transmissionstructure. For example, using the 5G transmission technology suitablefor high-definition videos, the HLS signal is transmitted to a server501 of a content delivery network (CDN) of a telecommunication serviceprovider, for a user to download the HLS signal in real time. Theprocess above is merely an example of a signal transmission mode androute of sending in real time a video from a “remote end” close to thereal site in the proximity of the first camera device 101, the secondcamera device 102, and the video processing device 200 to a “near end”in the proximity of a virtual reality user, so as to create a virtualreality space. It should be noted that the example does not imposelimitations on the technical solution of performing video transmissionthrough a non-physical wire above.

In some embodiments, the video generating system includes a virtualreality device 600. The virtual reality device 600 is for receiving astreaming signal, and decoding the streaming signal into the videooriginally outputted from the video processing device 200. To adapt tothe signal format restrictions of a physical wire, after the first video301 and the second video 302 that have undergone virtual realityprocessing are segmented and reorganized as above, they depart in theform of the third video 303 having a 16:9 aspect ratio from the videoprocessing device 200, and are converted into the streaming signal bythe streaming encoder 500 and eventually transmitted to the virtualreality device 600. At this time point, in order for the virtual realityuser to correctly read the video, the streaming signal is first decodedand again converted into the third video 303, and the decoded thirdvideo 303 needs to again undergo video segmenting and reorganizingbecause the third video 303 has a specific format for transmission.Thus, the third video 303 is restored/forms the first video 301 and thesecond video 302 again.

In some embodiments, the virtual reality device 600 includes a secondvideo processing unit 601 that may be a GPU. The virtual reality device600 may operate another shader (not shown) to generate an instructionfor the second video processing unit 601 to process and modify thepixels of the third video 303, which then is restored from theresolution in the schematic diagram of FIG. 4C to the resolution in theschematic diagram of FIG. 3 above. Since the second video processingunit 601 may be integrated into a consumer-grade wearable device,benefits including high efficiency and low power consumption incontribution to restoring the third video 303 to the first video 301 andthe second video 302 using the shader are further emphasized.

In some embodiments, after the third video 303 is reorganized into thevirtual reality video 30 having a 1:1 aspect ratio, the first video 301and the second video 302 in the virtual reality video 30 may besynchronously displayed by two display mechanisms in the virtual realitydevice 600, for example, a left-eye display mechanism 602L and aright-eye display mechanism 602R, allowing a virtual reality user tosense clear, three-dimensional, real-time and realistic live experience.Moreover, with customized special effects provided to the video or addedcontents such as data analysis information provided by a third party, ahigh standard sense of presence is made even more enjoyable.

In conclusion, in some embodiments of the disclosure, a video generatingsystem for virtual reality is provided. The video generating system forvirtual reality adjusts a plurality of original videos obtained bycamera devices to a plurality of virtual reality videos using a videoprocessing device for virtual reality provided by the disclosure.Moreover, in a video transmission method for virtual reality provided bythe disclosure, the virtual reality videos are reorganized to generate atransmission video having a 16:9 aspect ratio. Thus, withoutcompromising the resolution of the video, the transmission video mayefficiently meet video signal format requirements of a physical wire,thereby achieving high-resolution three-dimensional real-time virtualreality streaming services.

Structures of several embodiments are given above for a person skilledin the art to better understand the aspects of the disclosure. It isunderstandable by a person skilled in the art that, designs andmodifications may be easily made to other processes and structures onthe basis of the disclosure so as to achieve the same objects and/orbenefits of the embodiments described herein. Moreover, it is alsounderstandable by a person skilled in the art that, these equivalentstructures do not depart from the spirit and scope of the disclosure,and various changes, modifications and substitutions may also be made tothe details of the disclosure without departing from the spirit andscope of the disclosure.

What is claimed is:
 1. A video transmission method for virtual reality,the video transmission method comprising: obtaining a first video and asecond video, both the first video and the second video having a 2:1aspect ratio; segmenting the second video from a short side of thesecond video to generate two first sub videos having a same aspectratio; segmenting the first sub videos from a long side of one of thefirst sub videos to generate three second sub videos having a sameaspect ratio; generating a third video, wherein the third videocomprises the first video, the non-segmented first sub video and thethree second sub videos, and a short side of the third video is formedby a short side of the first video and a short side of the non-segmentedfirst sub video, and the other short side of the third video is formedby three short sides of the three second sub videos, such that the thirdvideo has a 16:9 aspect ratio; and transmitting the third video througha physical wire.
 2. The video transmission method of claim 1, wherein along side of the first video at least has 5,760 pixels.
 3. The videotransmission method of claim 1, wherein the operations of segmenting thesecond video, segmenting the first sub videos and generating the thirdvideo comprise operating a shader in a video processing device, suchthat the segmented second video and the segmented first sub videos arereorganized with the first video into the third video.
 4. The videotransmission method of claim 1, further comprising: transmitting thethird video to a streaming encoder through the physical wire, andconverting the third video into a streaming signal by the streamingencoder.
 5. The video transmission method of claim 4, furthercomprising: transmitting the streaming signal to a virtual realitydevice; decoding the streaming signal into the third video; andgenerating the first video and the second video using the third video.6. The video transmission method of claim 5, wherein the operation ofgenerating the first video and the second using the third videocomprises operating a shader in the virtual reality device, andreorganizing the third video and outputting the first video and thesecond video to two display mechanisms of the virtual reality device,respectively.
 7. The video transmission method of claim 1, before theoperation of segmenting the second video, further comprising: renderinga special effect to the first video and the second video.
 8. A methodfor processing video for non-compression transmission, the methodcomprising: receiving a first video and a second video, both the firstvideo and the second video having a 2:1 aspect ratio; segmenting thesecond video from a short side of the second video to generate two firstsub videos having a same aspect ratio; segmenting the first sub videosfrom a long side of one of the first sub videos to generate three secondsub videos having a same aspect ratio; and generating a third videocompatible with a transmission standard of a physical wire, wherein thethird video comprises the first video, the non-segmented first sub videoand the three second sub videos, and a short side of the third video isformed by a short side of the first video and a short side of thenon-segmented first sub video, and the other short side of the thirdvideo is formed by three short sides of the three second sub videos,such that the third video has a 16:9 aspect ratio.
 9. The method ofclaim 8, wherein the first video is obtained from a left-eyefield-of-view video captured by a video device.
 10. The method of claim9, wherein the second video is obtained from a right-eye field-of-viewvideo captured by the video device.
 11. The method of claim 8, whereinthe first video is stitched from two left-eye field-of-view videoscaptured by two video devices, respectively.
 12. The method of claim 11,wherein the second video is stitched from two right-eye field-of-viewvideos captured by the two video devices, respectively.
 13. The methodof claim 8, wherein the first video and the second video are 360-degreevirtual reality videos of different fields of view.
 14. The method ofclaim 8, further comprising: rendering a special effect to the firstvideo and the second video.
 15. The method of claim 8, wherein thephysical wire comprises Serial Digital Interface (SDI), High-DefinitionMultimedia Interface (HDMI) or DisplayPort (DP).
 16. The method of claim8, wherein a long side of the first video at least has 5,760 pixels. 17.The method of claim 16, wherein a short side of the first video at leasthas 2,880 pixels.
 18. The method of claim 8, wherein a pixel number ofthe third video is a total of a pixel number of the first video and apixel number of the second video.
 19. The method of claim 8, wherein thethird video has a 7680*4320 resolution.
 20. The method of claim 8,wherein the operations of segmenting the second video, segmenting thefirst sub videos, and generating the third video are performed by ashader.